Optimization methods for the insertion, protection, and detection of digital watermarks in digitized data

ABSTRACT

The implementations of digital watermarks can be optimally suited to particular transmission, distribution and storage mediums given the nature of digitally-sampled audio, video and other multimedia works. Watermark application parameters can be adapted to the individual characteristics of a given digital sample stream. Watermark information can be either carried in individual samples or in relationships between multiple samples, such as in a waveform shape. More optimal models may be obtained to design watermark systems that are tamper-resistant given the number and breadth of existent digitized sample options with different frequency and time components. The highest quality of a given content signal may be maintained as it is mastered, with the watermark suitably hidden, taking into account usage of digital filters and error correction. The quality of the underlying content signals can be used to identify and highlight advantageous locations for the insertion of digital watermarks. The watermark is integrated as closely as possible to the content signal, at a maximum level to force degradation of the content signal when attempts are made to remove the watermarks.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.08/677,435 filed Jul. 2, 1996, now U.S. Pat No. 5,889,868. Further, thisapplication is related to patent applications entitled “SteganographicMethod and Device”, Ser. No. 08/489,172 filed on Jun. 7, 1995; “Methodfor Human-Assisted Random Key Generation and Application for DigitalWatermark System”, Ser. No. 08/587,944 filed on Jan. 17, 1996; “Methodfor Stega-Cipher Protection of Computer Code”, Ser. No. 08/587,943 filedon Jan. 17, 1996; “Digital Information Commodities Exchange”, Ser. No.08/365,454 filed on Dec. 28, 1994, which is a continuation of Ser. No.08/083,593 filed on Jun. 30, 1993; and “Exchange Mechanisms for DigitalInformation Packages with Bandwidth Securitization, Multichannel DigitalWatermarks, and Key Management”, Ser. No. 08/674,726, filed on Jul. 2,1996. These related applications are all incorporated herein byreference.

This application is also related to U.S. Pat. No. 5,428,606, “DigitalInformation Commodities Exchange”, issued on Jun. 27, 1995, which isincorporated herein by reference.

BACKGROUND OF THE INVENTION

The present invention relates to digital watermarks.

Digital watermarks exist at a convergence point where creators andpublishers of digitized multimedia content demand localized, securedidentification and authentication of that content. Because existence ofpiracy is clearly a disincentive to the digital distribution ofcopyrighted works, establishment of responsibility for copies andderivative copies of such works is invaluable. In considering thevarious forms of multimedia content, whether “master,” stereo, NTSCvideo, audio tape or compact disc, tolerance of quality degradation willvary with individuals and affect the underlying commercial and aestheticvalue of the content. It is desirable to tie copyrights, ownershiprights, purchaser information or some combination of these and relateddata to the content in such a manner that the content must undergodamage, and therefore a reduction in value, with subsequent,unauthorized distribution of the content, whether it be commercial orotherwise.

Legal recognition and attitude shifts, which recognize the importance ofdigital watermarks as a necessary component of commercially distributedcontent (audio, video, game, etc.), will further the development ofacceptable parameters for the exchange of such content by the variousparties engaged in the commercial distribution of digital content. Theseparties may include artists, engineers, studios, INTERNET accessproviders, publishers, agents, on-line service providers, aggregators ofcontent for various forms of delivery, on-line retailers, individualsand parties that participate in the transfer of funds to arbitrate theactual delivery of content to intended parties.

Since the characteristics of digital recordings vary widely, it is aworthwhile goal to provide tools to describe an optimized envelope ofparameters for inserting, protecting and detecting digital watermarks ina given digitized sample (audio, video, virtual reality, etc.) stream.The optimization techniques described hereinafter make unauthorizedremoval of digital watermarks containing these parameters asignificantly costly operation in terms of the absolute given projectedeconomic gain from undetected commercial distribution. The optimizationtechniques, at the least, require significant damage to the contentsignal, as to make the unauthorized copy commercially worthless, if thedigital watermark is removed, absent the use of extremely expensivetools.

Presumably, the commercial value of some works will dictate some levelof piracy not detectable in practice and deemed “reasonable” by rightsholders given the overall economic return. For example, there willalways be fake $100 bills, LEVI jeans, and GUCCI bags, given the sizesof the overall markets and potential economic returns for pirates inthese markets—as there also will be unauthorized copies of works ofmusic, operating systems (Windows95, etc.), video and future multimediagoods.

However, what differentiates the “digital marketplace” from the physicalmarketplace is the absence of any scheme that establishes responsibilityand trust in the authenticity of goods. For physical products,corporations and governments mark the goods and monitor manufacturingcapacity and sales to estimate loss from piracy. There also existreinforcing mechanisms, including legal, electronic, and informationalcampaigns to better educate consumers.

SUMMARY OF THE INVENTION

The present invention relates to implementations of digital watermarksthat are optimally suited to particular transmission, distribution andstorage mediums given the nature of digitally-sampled audio, video, andother multimedia works.

The present invention also relates to adapting watermark applicationparameters to the individual characteristics of a given digital samplestream.

The present invention additionally relates to the implementation ofdigital watermarks that are feature-based. That is, a system wherewatermark information is not carried in individual samples, but iscarried in the relationships between multiple samples, such as in awaveform shape. The present invention envisions natural extensions fordigital watermarks that may also separate frequencies (color or audio),channels in 3D while utilizing discreteness in feature-based encodingonly known to those with pseudo-random keys (i.e., cryptographic keys)or possibly tools to access such information, which may one day exist ona quantum level.

The present invention additionally relates to a method for obtainingmore optimal models to design watermark systems that aretamper-resistant given the number and breadth of existentdigitized-sample options with differing frequency and time components(audio, video, pictures, multimedia, virtual reality, etc.).

To accomplish these goals, the present invention maintains the highestquality of a given content signal as it was mastered, with itswatermarks suitably hidden, taking into account usage of digital filtersand error correction presently concerned solely with the quality ofcontent signals.

The present invention additionally preserves quality of underlyingcontent signals, while using methods for quantifying this quality toidentify and highlight advantageous locations for the insertion ofdigital watermarks.

The present invention integrates the watermark, an information signal,as closely as possible to the content signal, at a maximal level, toforce degradation of the content signal when attempts are made to removethe watermarks.

The present invention relates to a method for amplitude independentencoding of digital watermark information in a signal including steps ofdetermining in the signal a sample window having a minimum and amaximum, determining a quantization interval of the sample window,normalizing the sample window, normalizing the sample window to providenormalized samples, analyzing the normalized samples, comparing thenormalized samples to message bits, adjusting the quantization level ofthe sample window to correspond to the message bit when a bit conflictswith the quantization level and de-normalizing the analyzed samples.

The present invention also relates to a method for amplitude independentdecoding of digital watermark information in a signal including steps ofdetermining in the signal a sample window having a minimum and amaximum, determining a quantization interval of the sample window,normalizing the sample window to provide samples, and analyzing thequantization level of the samples to determine a message bit value.

The present invention additionally relates to a method of encoding anddecoding watermarks in a signal where, rather than individual samples,insertion and detection of abstract signal features to carry watermarkinformation in the signal is-done.

The present invention also relates to a method for pre-analyzing adigital signal for encoding digital watermarks using an optimal digitalfilter in which it is determined what noise elements in the digitalsignal will be removed by the optimal digital filter based on responsecharacteristics of the filter.

The present invention also relates to a method of error coding watermarkmessage certificates using cross-interleaved codes which use error codesof high redundancy, including codes with Hamming distances of greaterthan or equal to “n”, wherein “n” is a number of bits in a messageblock.

The present invention additionally relates to a method of preprocessinga watermark message certificate including a step of determining anabsolute bit length of the watermark message as it will be encodedwatermark pseudo-random key bits using a non-linear (chaotic) generatoror to a method of mapping pseudo-random key and processing stateinformation to affect an encode/decode map using a non-linear (chaotic)generator.

The present invention additionally relates to a method of guaranteeingwatermark certificate uniqueness including a step of attaching a timestamp or user identification dependent hash or message digest ofwatermark certificate data to the certificate.

The present invention also relates to a method of generating andquantizing a local noise signal to contain watermark information wherethe noise signal is a function of at least one variable which depends onkey and processing state information.

The present invention also relates to a method of dithering watermarkquantizations such that the dither changes an absolute quantizationvalue, but does not change a quantization level or information carriedin the quantization.

The present invention further relates to a method of encoding watermarksincluding inverting at least one watermark bit stream and encoding awatermark including the inverted watermark bit stream.

The present invention also relates to a method of decoding watermarks byconsidering an original watermark synchronization marker, an invertedwatermark synchronization marker, and inverted watermarks, and decodingbased on those considerations.

The present invention also relates to a method of encoding and decodingwatermarks in a signal using a spread spectrum technique to encode ordecode where information is encoded or decoded at audible levels andrandomized over both frequency and time.

The present invention additionally relates to a method of analyzingcomposite digitized signals for watermarks including obtaining acomposite signal, obtaining an unwatermarked sample signal, timealigning the unwatermarked sample signal to the composite signal, gainadjusting the time aligned unwatermarked sample signal to the compositesignal, estimating a pre-composite signal using the composite signal andthe gain adjusted unwatermarked sample signal, estimating a watermarkedsample signal by subtracting the estimated pre-composite signal for thecomposite signal, and scanning the estimated watermark sample signal forwatermarks.

The present invention additionally relates to a method for varyingwatermark encode/decode algorithms automatically during the encoding ordecoding of a watermark including steps of (a) assigning a list ofdesired CODECs to a list of corresponding signal characteristics whichindicate use of particular CODECs, (b) during encoding/decoding,analyzing characteristics of the current sample frame in the signalstream, prior to delivering the frame to CODEC, (c) looking up thecorresponding CODEC from the list of CODECs in step (a) which matchesthe observed signal characteristics from step (b), (d) loading and/orpreparing the desired CODEC, (e) passing the sample frame to the CODECselected in step (c), and f) receiving the output samples from step (e).

The present invention also relates to a method for varying watermarkencode/decode algorithms automatically during the encoding or decodingof a watermark, including steps of (a) assigning a list of desiredCODECs to a list of index values which correspond to values computed tovalues computed as a function of the pseudo-random watermark key and thestate of the processing framework, (b) during encoding/decoding,computing the pseudo-random key index value for the current sample framein the signal stream, prior to delivering the frame to a CODEC, (c)looking up the corresponding CODEC from the list of CODECs in step (a)which matches the index value from step (b), (d) loading and/orpreparing the desired CODEC, (e) passing the sample frame to the CODECselected in step (c), and (f) receiving the output samples from step(e).

DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate two embodiments of the inventionand, together with a general description of the other embodimentsdiscussed herein, serve to explain the principles of the invention.

FIG. 1 depicts a process for amplitude independent encoding of digitalwatermark information in a signal in accordance with an embodiment ofthe invention, and FIG. 2 depicts a process for amplitude independentdecoding of digital watermark information in a signal in accordance withanother embodiment of the invention.

In particular, FIG. 1 depicts an embodiment of the invention thatincludes the steps of: selecting a sample window 101 having a minimumand a maximum, determining a quantization interval of the sample window102, normalizing the sample window 103 to provide normalized samples,analyzing the normalized samples 104, comparing the normalized samplesto message bits 105, adjusting the quantization level 106 of the samplewindow to correspond to the message bit when a bit conflicts with thequantization level and de-normalizing the analyzed samples 107.

FIG. 2 depicts the steps used for amplitude independent decoding ofdigital watermark information in a signal including steps of:determining in the signal a sample window 201 having a minimum and amaximum, determining a quantization interval of the sample window 202,normalizing the sample window 203 to provide samples, and analyzing thequantization level of the samples 204 to determine a message bit value.

DETAILED DESCRIPTION

The present invention relates to implementations of digital watermarksthat are optimally suited to particular transmission, distribution andstorage mediums given the nature of digitally sampled audio, video, andother multimedia works.

The present invention also relates to adapting watermark applicationparameters to the individual characteristics of a given digital samplestream.

The present invention additionally relates to the implementation ofdigital watermarks that are feature-based. That is, a system wherewatermark information is not carried in individual samples, but iscarried in the relationships between multiple samples, such as in awaveform shape. For example, in the same manner a US $100 bill has copyprotection features including ink type, paper stock, fiber, angles ofartwork that distort in photocopier machines, inserted magnetic strips,and composite art, the present invention envisions natural extensionsfor digital watermarks that may also separate frequencies (color oraudio), channels in 3D while utilizing discreteness in feature-basedencoding only known to those with pseudo-random keys (i.e.,cryptographic keys) or possibly tools to access such information, whichmay one day exist on a quantum level.

There are a number of hardware and software approaches in the prior artthat attempt to provide protection of multimedia content, includingencryption, cryptographic containers, cryptographic envelopes or“cryptolopes”, and trusted systems in general. None of these systemsplaces control of copy protection in the hands of the content creator asthe content is created, nor provides an economically feasible model forexchanging the content to be exchanged with identification data embeddedwithin the content.

Yet, given the existence of over 100 million personal computers and manymore non-copy-protected consumer electronic goods, copy protection seemsto belong within the signals. After all, the playing (i.e., using) ofthe content establishes its commercial value.

Generally, encryption and cryptographic containers serve copyrightholders as a means to protect data in transit between a publisher ordistributor and the purchaser of the data (i.e., a means of securing thedelivery of copyrighted material from one location to another by usingvariations of public key cryptography or other more centralizedcryptosystems).

Cryptolopes are suited specifically for copyrighted text that istime-sensitive, such as newspapers, where intellectual property rightsand origin data are made a permanent part of the file. For informationon public-key cryptosystems see U.S. Pat. No. 4,200,770 to Hellman etal., U.S. Pat. No. 4,218,582 to Hellman et al., U.S. Pat. No. 4,405,829to Rivest et al., and U.S. Pat. No. 4,424,414 to Hellman et al. Systemsare proposed by IBM and Electronic Publishing Resources to accomplishcryptographic container security.

Digitally-sampled copyrighted material, that is binary data on afundamental level, is a special case because of its long termvalue-coupled with the ease and perfectness of copying and transmissionby general purpose computing and telecommunications devices. Inparticular, in digitally-sampled material, there is no loss of qualityin copies and no identifiable differences between one copy and any othersubsequent copy. For creators of content, distribution costs may beminimized with electronic transmission of copyrighted works.Unfortunately, seeking some form of informational or commercial returnvia electronic exchange is ill-advised absent the use of digitalwatermarks to establish responsibility for specific copies andunauthorized copying. Absent digital watermarks, the unlikely instanceof a market of trusted parties who report any distribution or exchangeof unauthorized copies of the protected work must be relied upon forenforcement. Simply, content creators still cannot independently verifywatermarks should they choose to do so.

For a discussion of systems that are oriented around content-basedaddresses and directories, see U.S. Pat. No. 5,428,606 to Moskowitz.

In combining steganographic methods for insertion of informationidentifying the title, copyright holder, pricing, distribution path,licensed owner of a particular copy, or a myriad of other relatedinformation, with pseudo-random keys (which map insertion location ofthe information) similar to those used in cryptographic applications,randomly placed signals (digital watermarks) can be encoded as randomnoise in a content signal. Optimal planning of digital watermarkinsertion can be based on the inversion of optimal digital filters toestablish or map areas comprising a given content signal insertionenvelope. Taken further, planning operations will vary for differentdigitized content: audio, video, multimedia, virtual reality, etc.Optimization techniques for processes are described in the copendingrelated applications entitled “Steganographic Method and Device” and“Method for Human Assisted Random Key Generation and Application forDigital Watermark System”.

Optimization processes must take into consideration the general art ofdigitization systems where sampling and quantizing are fundamentalphysical parameters. For instance, discrete time sampling has a naturallimit if packets of time are used, estimated at 1×10⁻⁴² second. Thisprovides a natural limit to the sampling operation. Also, since noise ispreferable to distortion, quantizing will vary given different storagemediums (magnetic, optical, etc.) or transmission mediums (copper, fiberoptic, satellite, etc.) for given digitized samples (audio, video,etc.). Reducing random bit error, quantization error, burst error, andthe like is done for the singular goal of preserving quality in a givendigitized sample. Theoretical perfect error correction is not efficient,given the requirement of a huge allocation of redundant data to detectand correct errors. In the absence of such overhead, all errorcorrection is still based on data redundancy and requires the followingoperations: error detection to check data validity, error correction toreplace erroneous data, and error concealment to hide large errors orsubstitute data for insufficient data correction. Even with perfecterror correction, the goal of a workable digital watermark system forthe protection of copyrights would be to distribute copies that are lessthan perfect but not perceivably different from the original.Ironically, in the present distribution of multimedia, this is theapproach taken by content creators when faced with such distributionmechanisms as the INTERNET. As an example, for audio clips commerciallyexchanged on the World Wide Web (WWW), a part of the INTERNET, 8 bitsampled audio or audio downsampled from 44.1 kHz (CD-quality), to 22 kHzand lower. Digital filters, however, are not ideal because of trade-offsbetween attenuation and time-domain response, but provide the engineeror similarly-trained individual with a set of decisions to make aboutmaximizing content quality with minimum data overhead and considerationof the ultimate delivery mechanism for the content (CDs, cabletelevision, satellite, audio tape, stereo amplifier, etc.).

For audio signals and more generally for other frequency-based content,such as video, one method of using digital filters is to include the useof an input filter to prevent frequency aliasing higher than theso-called Nyquist frequencies. The Nyquist theorem specifies that thesampling frequency must be at least twice the highest signal frequencyof the sampled information (e.g., for the case of audio, humanperception of audio frequencies is in a range between 20 Hz and 20 kHz).Without an input filter, aliases can still occur leaving an aliasedsignal in the original bandwidth that cannot be removed.

Even with anti-aliasing filters, quantization error can still cause lowlevel aliasing which may be removed with a dither technique. Dither is amethod of adding random noise to the signal, and is used to de-correlatequantization error from the signal while reducing the audibility of theremaining noise. Distortion may be removed, but at the cost of addingmore noise to the filtered output signal. An important effect is thesubsequent randomization of the quantization error while still leavingan envelope of an unremovable signaling band of noise. Thus, dither isdone at low signal levels, effecting only the least significant bits ofthe samples. Conversely, digital watermarks, which are essentiallyrandomly-mapped noise, are intended to be inserted into samples ofdigitized content in a manner such as to maximize encoding levels whileminimizing any perceivable artifacts that would indicate their presenceor.allow for removal by filters, and without destroying the contentsignal. Further, digital-watermarks should be inserted with processesthat necessitate random searching in the content signal for watermarksif an attacker lacks the keys. Attempts to over-encode noise into knownwatermarked signal locations to eliminate the information signal can bemade difficult or impossible without damaging the content signal byrelying on temporal encoding and randomization in the generation of keysduring digital watermark insertion. As a result, although the watermarkoccupies only a small percentage of the signal, an attacker is forced toover-encode the entire signal at the highest encoding level, whichcreates audible artifacts.

The present invention relates to methods for obtaining more optimalmodels to design watermark systems that are tamper-resistant given thenumber and breadth of existent digitized sample options with differingfrequency and time components (audio, video, pictures, multimedia,virtual reality, etc.).

To accomplish these goals, the present invention maintains the highestquality of a given content signal as it was mastered, with itswatermarks suitably hidden, taking into account usage of digital filtersand error correction presently concerned solely with the quality ofcontent signals.

Additionally, where a watermark location is determined in a random orpseudo-random operation dependent on the creation of a pseudo-randomkey, as described in copending related application entitled“Steganographic Method and Device” assigned to the present assignee, andunlike other forms of manipulating digitized sample streams to improvequality or encode known frequency ranges, an engineer seeking to providehigh levels of protection of copyrights, ownership, etc. is concernedwith the size of a given key, the size of the watermark message and themost suitable area and method of insertion. Robustness is improvedthrough highly redundant error correction codes and interleaving,including codes known generally as q-ary Bose-Chaudhuri-Hocquenghem(BCH) codes, a subset of Hamming coding operations, and codes combiningerror correction and interleaving, such as the Cross-interleaveReed-Solomon Code. Using such codes to store watermark information inthe signal increases the number of changes required to obliterate agiven watermark. Preprocessing the certificate by considering errorcorrection and the introduction of random data to make watermarkdiscovery more difficult, prior to watermarking, will help determinesufficient key size. More generally, absolute key size can be determinedthrough preprocessing the message and the actual digital watermark (afile including information regarding the copyright owner, publisher, orsome other party in the chain of exchange of the content) to compute theabsolute encoded bit stream and limiting or adjusting the key sizeparameter to optimize the usage of key bits. The number of bits in theprimary key should match or exceed the number of bits in the watermarkmessage, to prevent redundant usage of key bits. Optimally, the numberof bits in the primary key should exactly match the watermark size,since any extra bits are wasted computation.

Insertion of informational signals into content signals and ranges fromapplications that originate in spread spectrum techniques have beencontemplated. More detailed discussions are included in copendingrelated applications entitled “Steganographic Method and Device” andentitled “Method for Human Assisted Random Key Generation andApplication for Digital Watermark System”.

The following discussion illustrates some previously disclosed systemsand their weaknesses.

Typically, previously disclosed systems lack emphasis or implementationof any pseudo-random operations to determine the insertion location, ormap, of information signals relating to the watermarks. Instead,previous implementations provide “copy protect” flags in obvious,apparent and easily removable locations. Further, previousimplementations do not emphasize the alteration of the content signalupon removal of the copy protection.

Standards for digital audio tape (DAT) prescribe insertion of data suchas ISRC (Industry Standard Recording Codes) codes, title, and time insub-code according to the Serial Copy Management System (SCMS) toprevent multiple copying of the content. One time copying is permitted,however, and systems with AES3 connectors, which essentially overridecopy protection in the sub-code as implemented by SCMS, actually have nocopy limitations. The present invention provides improvement over thisimplementation with regard to the ability of unscrupulous users to loaddigital data into unprotected systems, such general computing devices,that may store the audio clip in a generalized file format to bedistributed over an on-line system for further duplication. The securityof SCMS (Serial Copy Management System) can only exist as far as thesupport of similarly-oriented hardware and the lack of attempts by thoseskilled in the art to simply remove the subcode data in question.

Previous methods seek to protect content, but shortcomings are apparent.U.S. Pat. No. 5,319,735 to Preuss et al. discusses a spread spectrummethod that would allow for over-encoding of the described, thus known,frequency range and is severely limited in the amount of data that canbe encoded—4.3 8-bit symbols per second. However, with the Preuss et al.method, randomization attacks will not result in audible artifacts inthe carrier signal, or degradation of the content as the informationsignal is in the subaudible range. It is important to note thedifference in application between spread spectrum in military field usefor protection of real-time radio signals, and encoding information intostatic audio files. In the protection of real-time communications,spread spectrum has anti-jam features, since information is sent overseveral channels at once. Therefore, in order to jam the signal, one hasto jam all channels, including their own. In a static audio file,however, an attacker has practically unlimited time and processing powerto randomize each sub-channel in the signaling band without penalty tothemselves, so the anti-jam advantages of spread spectrum do not extendto this domain.

In a completely different implementation, U.S. Pat. No. 5,379,345 toGreenberg seeks enforcement of broadcast contracts using a spreadspectrum modulator to insert signals that are then confirmed by a spreadspectrum-capable receiver to establish the timing and length that agiven, marked advertisement is played. This information is measuredagainst a specific master of the underlying broadcast material. TheGreenberg patent does not ensure that real-time downloads of copyrightedcontent can be marked with identification information unless alldownload access points (PCs, modems, etc.), and upload points for thatmatter, have spread spectrum devices for monitoring.

Other methods include techniques similar to those disclosed in relatedcopending patent applications mentioned above by the present assignee,but lack the pseudo-random dimension of those patent applications forsecuring the location of the signals inserted into the content. Oneimplementation conducted by Michael Gerzon and Peter Craven, anddescribed by Ken Pohlmann in the 3rd edition of Principles of DigitalAudio, illustrates a technology called “buried data technique,” but doesnot address the importance of randomness in establishing the insertionlocations of the informational signals in a given content signal, as nopseudo-random methods are used as a basis for insertion. The overridingconcern of the “buried data techniques” appears to be to provide for a“known channel” to be inserted in such a manner as to leave little or noperceivable artifacts in the content signal while prescribing the exactlocation of the information (i.e., replacing the least significant bits(LSB) in a given information signal). In Gerzon and Craven's example, a20-bit signal gives way to 4-bits of LSBs for adding about 27 dB ofnoise to the music. Per channel data insertion reached 176.4 kilobitsper second per channel, or 352.8 kbps with stereo channels. Similarlyattempted data insertion by the present inventors using random datainsertion yielded similar rates. The described techniques may beinvaluable to manufacturers seeking to support improvements in audio,video and multimedia quality improvements. These include multiple audiochannel support, surround sound, compressed information on dynamicrange, or any combination of these and similar data to improve quality.Unfortunately, this does little or nothing to protect the interests ofcopyright holders from unscrupulous pirates, as they attempt to createunmarked, perfect copies of copyrighted works.

The present invention also relates to copending patent applicationsentitled “Staganographicc Method and Device”; “Method for Human-AssistedRandom Key Generation and Application for Digital Watermark System”; and“Method for Stega-Cipher Protection of Computer Code” as mentionedabove, specifically addressing the weakness of inserting informationalsignals or digital watermarks into known locations or known frequencyranges, which are sub-audible. The present invention seeks to improve onthe methods disclosed in these patent applications and other methods bydescribing specific optimization techniques at the disposal of thoseskilled in the art. These techniques provide an a la carte method forrethinking error correction, interleaving, digital and analog filters,noise shaping, nonlinear random location mapping in digitized samples,hashing, or making unique individual watermarks, localized noise signalmimic encoding to defeat noise filtering over the entire sample stream,super audible spread spectrum techniques, watermark inversion,preanalyzing watermark key noise signatures, and derivative analysis ofsuspect samples against original masters to evaluate the existence ofwatermarks with statistical techniques.

The goal of a digital watermark system is to insert a given informationsignal or signals in such a manner as to leave few or no artifacts inthe underlying content signal, while maximizing its encoding level andlocation sensitivity in the signal to force damage to the content signalwhen removal is attempted. The present invention establishes methods,for estimating and utilizing parameters, given principles of thedigitization of multimedia content (audio, video, virtual reality,etc.), to create an optimized “envelope” for insertion of watermarks,and thus establish secured responsibility for digitally sampled content.The pseudo-random key that is generated is the only map to access theinformation signal while not compromising the quality of the content. Adigital watermark naturally resists attempts at removal because itexists as purely random or pseudo-random noise in a given digitizedsample. At the same time, inversion techniques and mimicking operations,as well as encoding signal features instead of given samples, can makethe removal of each and every unique encoded watermark in a givencontent signal economically infeasible (given the potential commercialreturns of the life of a given copyright) or impossible withoutsignificantly degrading the quality of the underlying, “protected”signal. Lacking this aesthetic quality, the marketability or commercialvalue of the copy is correspondingly reduced.

The present invention preserves quality of underlying content signals,while using methods for quantifying this quality to identify andhighlight advantageous locations for the insertion of digitalwatermarks.

The present invention integrates the watermark, an information signal,as closely as possible to the content signal, at a maximal level, toforce degradation of the content signal when attempts are made to removethe watermarks.

General methods for watermarking digitized content, as well as computercode, are described in copending related patent applications entitled“Steganographic Method and Device” and entitled “Method for Stega-CipherProtection of Computer Code”, both assigned to the present assignee.Recognizing the importance of perceptual encoding of watermarks by theauthors and engineers who actually create content is addressed incopending related application entitled “Method for Human Assisted RandomKey Generation and Application for Digital Watermark System”.

The present invention describes methods of random noise creation giventhe necessary consequence of improving signal quality with digitizationtechniques. Additionally, methods are described for optimizingprojections of data redundancy and overhead in error correction methodsto better define and generate parameters by which a watermarking systemcan successfully create random keys and watermark messages thatsubsequently cannot be located and erased without possession of the keythat acts as the map for finding each encoded watermark. Thisdescription will provide the backdrop for establishing truly optimizedwatermark-insertion including: use of nonlinear (chaotic) generators;error correction and data redundancy analysis to establish a system foroptimizing key and watermark message length; and more general issuesregarding desired quality relating to the importance of subjectingwatermarked content to different models when the content may bedistributed or sold in a number of prerecorded media formats ortransmitted via different electronic transmission systems; this includesthe use of perceptual coding; particularized methods such as noiseshaping; evaluating watermark noise signatures for predictability;localized noise function mimic encoding; encoding signal features;randomizing time to sample encoding of watermarks; and, finally, astatistical method for analyzing composite watermarked content against amaster sample content to allow watermark recovery. All of these featurescan be incorporated into specialized digital signal processingmicroprocessors to apply watermarks to nongeneralized computing devices,such as set-top boxes, video recorders that require time stamping orauthentication, digital video disc (DVD) machines and a multitude ofother mechanisms that play or record copyrighted content.

The sampling theorem, known specifically as the Nyquist Theorem, provesthat bandlimited signals can be sampled, stored, processed, transmitted,reconstructed, desampled or processed as discrete values. In order forthe theorem to hold true, the sampling must be done at a frequency thatis at least twice the frequency of the highest signal frequency to becaptured and reproduced. Aliasing will occur as a form of signal foldover, if the signal contains components above the Nyquist frequency. Toestablish the highest possible quality in a digital signal, aliasing isprevented by low-pass filtering the input signal to a given digitizationsystem by a low-pass or anti-aliasing filter. Any residue aliasing whichmay result in signal distortion relates to another area of signalquality control, namely, quantization error removal.

Quantization is required in a digitization system. Because of thecontinuous nature of an analog signal (amplitude vs. time), a quantizedsample of the signal is an imperfect estimate of the signal sample usedto encode it as a series of discrete integers. These numbers are merelyestimates of the true value of the signal amplitude. The differencebetween the true analog value at a discrete time and the quantizationvalue is the quantization error. The more bits allowed per sample, thegreater the accuracy of estimation; however, errors still always willoccur. It is the recurrent nature of quantization errors that providesan analogy with the location of digital watermarks.

Thus, methods for removal of quantization errors have relevance inmethods for determining the most secure locations for placement ofwatermarks to prevent the removal of such watermarks.

The highest fidelity in digital reproduction of a signal occurs atpoints where the analog signal converges with a given quantizationinterval. Where there is no such convergence, in varying degrees, thequantization error will be represented by the following range:

+Q/2 and −Q/2, where Q is the quantization interval.

Indeed, describing maximization of the quantization error and its ratiowith the maximum signal amplitude, as measured, will yield asignal-to-error ratio (S/E) which is closely related to the analogsignal-to-noise ratio (S/N). To establish more precise boundaries fordetermining the S/E, with root mean square (rms) quantization errorE_(rms), and assuming a uniform probability density function 1/Q(amplitude), the following describes the error:

E _(rms) =Q/(12)^(½)

Signal to quantization error is expressed as:

S/E=[S _(rms) /E _(rms)]²=3/2(2^(2n))

Finally, in decibels (dB) and comparing 16-bit and 15-bit quantization:

S/E(dB)=10 log[3/2(2^(2n))]=10 log 3/2+2^(n) log 2

(or “=20 log[(3/2)^(½)(2^(n))]”)

=6.02n+1.76

This explains the S/E ratio of 98 dB for 16-bit and 92 dB for 15-bitquantization. The 1.76 factor is established statistically as a resultof peak-to-rms ratio of a sinusoidal waveform, but the factor willdiffer if the signal waveform differs. In complex audio signals, anydistortion will exist as white noise across the audible range. Lowamplitude signals may alternatively suffer from distortion.

Quantization distortion is directly related with the original signal andis thus contained in the output signal, it is not simply an error. Thisbeing the case, implementation of so-called quality control of thesignal must use dither. As discussed above, dither is a method of addingrandom noise to the signal to de-correlate quantization error from thesignal while reducing the audibility of the remaining noise. Distortionmay be removed at the cost of adding more noise to the filtered outputsignal. An important effect is the subsequent randomization of thequantization error while still leaving an envelope of an unremovablesignaling band of noise. Dither, done at low signal levels, effects onlythe least significant bits of the samples.

Use of linear and nonlinear quantization can effect the trade-off in theoutput signal and must be considered for a system of watermarks designedto determine acceptable quantization distortion to contain the digitalwatermark. For audio systems, block linear quantization implementationshave been chosen. However, block floating point and floating pointsystems, nonuniform companding, adaptive delta modulation, adaptivedifferential pulse-code modulation, and perceptual coding schemes (whichare oriented around the design of filters that closely match the actualperception of humans) appear to provide alternative methodimplementations that would cause higher perceptible noise artifacts iffiltering for watermarks was undertaken by pirates. The choice of methodis related to the information overhead desired.

According to one aspect of the present invention, the envelope describedin the quantization equations above is suitable for preanalysis of adigitized sample to evaluate optimal locations for watermarks. Thepresent example is for audio, but corresponding applications fordigitization of video would be apparent in the quantization of colorfrequencies.

The matter of dither complicates preanalysis of a sample evaluated fordigital watermarks. Therefore, the present invention also defines theoptimal envelope more closely given the three types of dither (thisexample is for audio, others exist for video): triangular probabilitydensity function (pdf, Gaussian pdf, and rectangular pdf. Again, toestablish better boundaries for the random or pseudo-random insertion ofa watermark to exist in a region of a content signal that wouldrepresent an area for hiding watermarks in a manner most likely to causedamage to the content signal if unauthorized searches or removal areundertaken. Dither makes removal of quantization error more economicalthrough lower data overhead in a system by shifting the signal range todecorrelate errors from the underlying signal. When dither is used, thedither noise and signal are quantized together to randomize the error.Dither which is subtractive requires removing the dither signal afterrequantization and creates total error statistical independence. Itwould also provide further parameters for digital watermark insertiongiven the ultimate removal of the dither signal before finalizing theproduction of the content signal. With nonsubtractive dither, the dithersignal is permanently left in the content signal. Errors would not beindependent between samples. For this reason, further analysis with thethree types of dither should reveal an acceptable dither signal withoutmaterially affecting the signal quality.

Some proposed systems for implementing copyright protection intodigitally-sampled content, such as that proposed by DigimarcCorporation, predicate the natural occurrence of artifacts that cannotbe removed. Methods for creating a digital signature in the minimizederror that is evident, as demonstrated by explanations of dither, pointout another significant improvement over the art in the system describedin the present invention and its antecedents. Every attempt is made toraise the error level of error from LSBs to a level at which erasurenecessarily leads to the degradation of the “protected” content signal.Furthermore, with such a system, pirates are forced to make guesses, andthen changes, at a high enough encoding level over a maximum amount ofthe content signal so as to cause signal degradation, because guessingnaturally introduces error. Thus, dither affects the present invention'senvelope by establishing a minimum encoding level. Any encoding donebelow the dither level might be erased by the dither.

One embodiment of the present invention may be viewed as the provisionof a random-super-level non-subtractive dither which containsinformation (the digital watermark).

To facilitate understanding of how this does not cause audibleartifacts, consider the meaning of such encoding in terms of the S/Eratio. In a normal 16-bit signal, there is a 98 dB S/E according to theequation S/E=6.02n+1.76. Consider that the encoding of watermarkinformation looks like any other error, except it moves beyond thequantization level, out of the LSBs. If the error is of a magnitudeexpressed in, say, 8 bits, then at that moment, the signal effectivelydrops to 8 bits (16-8). This corresponds to a momentary drop in S/E,referred to herein as the momentary S/E. Yet, these errors arerelatively few and far between and therefore, since the signal isotherwise comprised of higher-bit samples, a “Perceived S/E” may bederived which is simply the weighted average of the samples using the“Pure S/E” (the samples without watermark information) and those withthe Momentary S/E. As a direct consequence, it may be observed that themore sparse the watermark map, the fewer errors introduced in a givenrange, and the higher the perceived S/E. It also helps that the error israndom, and so over time, appears as white noise, which is relativelyunobtrusive. In general, it is observed that as long as introducederrors leave resulting samples within an envelope in the sample windowdescribed by minimum and maximum values, before error introduction, andthe map is sufficiently sparse, the effects are not perceived.

In addition, it is possible to obtain an even higher Perceived S/E byallowing the range of introduced errors to vary between a minimum andmaximum amount. This makes the weighted average S/E higher by reducingthe average introduced error level. Yet, someone trying to erase awatermark, assuming they knew the maximum level, would have to erase atthat level throughout the data, since they would not know how theintroduced level varies randomly, and would want to erase allwatermarks.

A watermarking cipher could perform this operation and may alsointroduce the further step of local dither (or other noise)significantly above the quantization amplitude on a window by windowbasis randomly, to restrict total correlation between the watermarksignal and the probability that it remains independent between samples,as with subtractive dither implementations that are mostly concernedwith the ultimate removal of the dither signal with requantization. Thisability could be used to accomplish signal doping, which adds a degreeof random errors that do not contain watermark information so as toprevent differential analysis of multiple watermarked copies.Alternatively, it could be used to mimic a specific noise function in asegment of the signal in order to defeat attempts to filter a particulartype of noise over the entire signal. By varying this function betweenwatermarks it may be guaranteed that any particular filter is of no useover the whole signal. By applying several filters in series, it seemsintuitive that the net results would be significantly different from theoriginal signal.

The discussion may be more appropriately introduced with perceptualcoding techniques, but a watermarking system could also defeat somedetection and correction with dither by inserting watermarks into signalfeatures, instead of signal samples. This would be equivalent to lookingfor signal characteristics, independent of the overall sample as itexists as a composite of a number of signals. Basically, instead ofencoding on a bit per sample basis, one might spread bits over severalsamples. The point of doing this is that filtering and convolutionoperations, like “flanging”, which definitely change individual sampleson a large scale, might leave intact enough of a recognizable overallsignal structure (the relationship between multiple samples) to preservethe watermark information. This may be done by measuring, generalizing,and altering features determined by the relationships between samples orfrequency bands. Because quantization is strictly an art ofapproximation, signal-to-error ratios, and thus the dynamic range of agiven system are determined.

The choice of eliminating quantization distortion at the expense ofleaving artifacts (not perceptible) is a permanent trade-off evident inall digitization systems which are necessarily based on approximation(the design goal of the present invention in preanalyzing a signal tomask the digital watermarks make imperceptibility possible). The highfidelity of duplication and thus subsequent ability to digitally orelectronically transmit the finished content (signal) is favored byconsumers and artists alike. Moreover, where there continues to be aquestion of approximating in quantization—digital watermark systems willhave a natural partner in seeking optimized envelopes in the multitudeand variety of created digitized content.

Another aspect of optimizing the insertion of digital watermarks regardserror correction. Highly redundant error codes and interleaving mightcreate a buffer against burst errors introduced into digital watermarksthrough randomization attacks. A detailed description follows from thenature of a digitization system—binary data can be corrected orconcealed when errors exist. Random bit errors and burst errors differin their occurrence:

Random bit errors are error bits occurring in a random manner, whereasburst errors may exist over large sequences of the binary datacomprising a digitized signal. Outside the scope of the presentinvention are errors caused by physical objects, such as dust andfingerprints, that contribute to the creation of dropouts are differentfrom the errors addressed herein.

Measuring error with bit-error ratio (BER), block error ratio (BLER) andburst-error length (BERL), however, provides the basis of errorcorrection. Redundancy of data is a focus of the present invention. Thisdata necessarily relies on existing data, the underlying content. Toefficiently describe optimal parameters for generating a cryptographickey and the digital watermark message discussion of error correction anderror concealment techniques is important.

Forms of error detection include one-bit parity, relying on themathematical ability to cast out numbers, for binary systems includingdigitization systems, such as 2. Remainders given odd or even results(parity) that are probablistically determined to be errors in the data.For more appropriate error detection algorithms, such as CyclicRedundancy Check Code (CRCC), which are suited for the detection ofcommonly occurring burst error. Pohlmann (Principles of Digital Audio)notes the high accuracy of CRCC (99.99%) and the truth of the followingstatements given a k-bit data word with m bits of CRCC, a code word of nbits is formed (m=n−k):

burst errors less than or equal to m bits are always predictable.

the detection probability of burst errors of m+1 bits=1−2^(−m+1).

the detection probability of burst errors longer than m+1 bits=1−2^(−m).

random errors up to 3 consecutive bits long can be detected.

The medium of content delivery, however, provides the ultimate floor forCRCC design and the remainder of the error correction system.

Error correction techniques can be broken into three categories: methodsfor algebraic block codes, probablistic methods for convolutional codes,and cross-interleave code where block codes are used in a convolutionstructure. As previously discussed, the general class of coces thatassist in pointing out the location of error are known generally asHamming codes, versus CRCC which is a linear block code.

What is important for establishing parameters for determining optimizederror coding in systems such as digital audio are more specificallyknown as Reed-Solomon Codes which are effective methods for correctingburst errors. Certain embodiments of the present invention presupposethe necessity of highly redundant error codes and interleaving, such asthat done in Cross Interleave Reed-Solomon Code, to counter burst errorstypically resulting from randomization attacks. More generally, certainembodiments of the present invention include the use of Hamming Codes of(n,n) to provide n−1 bit error detection and n−2 bit error correction.Further, a Hamming distance of n (or greater than n) is significantbecause of the nature of randomization attacks. Such an attack seeks torandomize the bits of the watermark message. A bit can be either 0 or 1,so any random change has a 50% chance of actually changing a bit fromwhat it was (50% is indicative of perfect randomness). Therefore, onemust assume that a good attack will change approximately half the bits(50%). A Hamming distance of n or greater, affords redundancy on a closepar with such randomization. In other words, even if half the bits arechanged, it would still be possible to recover the message.

Because interleaving and parity makes data robust for error avoidance,certain embodiments of the present invention seek to perform timeinterleaving to randomly boost momentary S/E ratio and give a betterestimate of not removing keys and watermarks that may be subsequentlydetermined to be “errors.”

Given a particular digital content signal, parity, interleaving, delay,and cross-interleaving, used for error correction, should be taken intoaccount when preprocessing information to compute absolute sizerequirements of the encoded bit stream and limiting or adjusting keysize parameters to optimize and perhaps further randomize usage of keybits. In addition, these techniques minimize the impact of errors andare thus valuable in creating robust watermarks.

Uncorrected errors can be concealed in digital systems. Concealmentoffers a different dynamic to establish insertion parameters for thepresent invention. Error concealment techniques exist because it isgenerally more economical to hide some errors instead of requiringoverly expensive encoders and decoders and huge information overheads indigitization systems. Muting, interpolation, and methods for signalrestoration (removal of noise) relate to methods suggested by thepresent invention to invert some percentage or number of watermarks soas to ensure that at least some or as many as half of the watermarksmust still remain in the content signal to effectively eliminate theother half. Given that a recording contains noise, whether due towatermarks or not, a restoration which “removes” such noise is likely toresult in the changing of some bit of the watermark message. Therefore,by inverting every other watermark, it is possible to insure that thevery act of such corrections inverts enough watermark bits to create aninverse watermark. This inversion presupposes that the optimizedwatermark insertion is not truly optimal, given the will of a determinedpirate to remove watermarks from particularly valuable content.

Ultimately, the inability to resell or openly trade unwatermarkedcontent will help enforce, as well as dictate, the necessity ofwatermarked content for legal transactions.

The mechanisms discussed above reach physical limits as the intent ofsignal filtering and error correction are ultimately determined to beeffective by humans—decidedly analog creatures. All output devices arethus also analog for playback.

The present invention allows for a preprocessed and preanalyzed signalstream and watermark data to be computed to describe an optimizedenvelope for the insertion of digital watermarks and creation of apseudo-random key, for a given digitized sample stream. Randomizing thetime variable in evaluating discrete sample frames of the content signalto introduce another aspect of randomization could further thesuccessful insertion of a watermark. More importantly, aspects ofperceptual coding are suitable for methods of digital watermarks orsuper-audible spread spectrum techniques that improve on the artdescribed by the Preuss et al. patent described above.

The basis for a perceptual coding system, for audio, is psychoacousticsand the analysis of only what the human ear is able to perceive. Similaranalysis is conducted for video systems, and some may argue abused, withsuch approaches as “subliminal seduction” in advertising campaigns.Using the human for design goals is vastly different than describingmathematical or theoretical parameters for watermarks. On some level ofdigital watermark technology, the two approaches may actually complementeach other and provide for a truly optimized model.

The following example applies to audio applications. However, thisexample and other examples provided herein are relevant to video systemsas well as audio systems. Where a human ear can discern between energyinside and outside the “critical band,” (described by Harvey Fletcher)masking can be achieved. This is particularly important as quantizationnoise can be made imperceptible with perceptual coders given themaintenance of a sampling frequency, decreased word length (data) basedon signaling conditions. This is contrasted with the necessary decreaseof 6 dB/bit with decreases in the sampling frequency as described abovein the explanation of the Nyquist Theorem. Indeed, data quantity can bereduced by 75%. This is an extremely important variable to feed into thepreprocessor that evaluates the signal in advance of “imprinting” thedigital watermark.

In multichannel systems, such as MPEG-1, AC-3 and other compressionschemes, the data requirement (bits) is proportional to the square rootof the number of channels. What is accomplished is masking that isnonexistent perceptually, only acoustically.

Taken to another level for digital watermarking, which is necessary forcontent that may be compressed and decompressed, forward adaptiveallocation of bits and backward adaptive allocation provide for encodingsignals into content signals in a manner such that information can beconveyed in the transmission of a given content signal that issubsequently decoded to convey the relatively same audible signal to asignal that carries all of its bits—e.g., no perceptual differencesbetween two signals that differ in bit size. This coding technique mustalso be preanalyzed to determine the most likely sample bits, or signalcomponents, that will exist in the smaller sized signal. This is alsoclearly a means to remove digital watermarks placed into LSBs,especially when they do not contribute theoretically perceptible valueto the analyzed signal. Further methods for data reduction coding aresimilarly important for preanalyzing a given content signal prior towatermarking. Frequency domain coders such as subband and transformbands can achieve data reduction of ratios between 4:1 and 12:1. Thecoders adaptively quantize samples in each subband based on the maskingthreshold in that subband (See Pohlmann, Principles of Digital Audio).Transform coders, however, convert time domain samples into thefrequency domain for accomplishing lossless compression. Hybrid coderscombine both subband and transform coding, again with the ultimate goalof reducing the overall amount of data in a given content signal withoutloss of perceptible quality.

With digital watermarks, descriptive analysis of an information signalis important to preanalyze a given watermark's noise signature. Analysisof this signature versus the preanalysis of the target content signalfor optimized insertion location and key/message length, are potentiallyimportant components to the overall implementation of a securewatermark. It is important that the noise signature of a digitalwatermark be unpredictable without the pseudo-random key used to encodeit. Noise shaping, thus, has important applications in theimplementation of the present invention. In fact, adaptive dithersignals can be designed to correlate with a signal so as to mask theadditional noise—in this case a digital watermark. This relates to theabove discussion of buried data techniques and becomes independentlyimportant for digital watermark systems. Each instance of a watermark,where many are added to a given content signal given the size of thecontent and the size of the watermark message, can be “noise shaped” andthe binary description of the watermark signature may be made unique by“hashing” the data that comprises the watermark. Generally, hashing thewatermark certificate prior to insertion is recommended to establishdifferences between the data in each and every watermark “file.”

Additionally, the present invention provides a framework in which toanalyze a composite content signal that is suspected to contain awatermarked sample of a copyrighted work, against an unwatermarkedoriginal master of the same sample to determine if the composite contentactually contains a copy of a previously watermarked content signal.Such an analysis may be accomplished in the following scenario:

Assume the composite signal contains a watermark from the sample.

Assume the provision of the suspect composite signal C_(w)(t) (wsubscript denotes a possible watermark) and the unwatermarked originalsample S_(uw)(t). These are the only two recordings the analyzer islikely to have access to.

Now, it is necessary to recover a watermarked sample S_(w)(t).

The methods of digital signal processing allow for the computation of anoptimal estimate of a signal. The signal to be estimated is thecomposite minus the watermarked sample, or C″_(w)(t)=C_(w)(t)−S_(w)(t).The analyzer, however, cannot determine a value of S_(w)(t), since itdoes not know which of the many possible S_(w)(t) signals was used inthe composite. However, a close estimate may be obtained by usingS_(uw)(t), since watermarking makes relatively minor changes to asignal.

So, C″_(w)(t) (an estimate of C′_(w)(t) given C_(w)(t) and S_(uw)(t))may be obtained. Once C″_(w)(t) is calculated, it is simply subtractedfrom C_(w)(t). This yields S′_(w)(t)=C_(w)(t)−C″_(w)(t). If thewatermark is robust enough, and the estimate good enough, thenS′_(w)(t), which is approximately equal to S_(w)(t), can be processed toextract the watermark. It is simply a matter of attempting watermarkdecoding against a set of likely encoding key candidates.

Note that although a watermark is initially suspected to be present inthe composite, and the process as if it is, the specifics of thewatermark are not known, and a watermark is never introduced into thecalculations, so a watermark is extracted, it is valid, since it was notintroduced by the signal processing operations.

The usefulness of this type of operation is demonstrated in thefollowing scenario:

People are interested in simply proving that their copyrighted samplewas dubbed into another recording, not the specifics of ownership of thesample used in the dubbing. So, this implies that only a single, orlimited number of watermark keys would be used to mark samples, andhence, the decode key candidates are limited, since the same key wouldbe used to encode-simple copyright information which never varies fromcopy to copy.

There are some problems to solve to accomplish this sort of processing.The sample in question is generally of shorter duration than thecomposite, and its amplitude may be different from the original.Analysis techniques could use a combination of human-assisted alignmentin the time domain, where graphical frequency analysis can indicate thetemporal location of a signal which closely matches that of the originalsample. In addition, automatic time warping algorithms which time alignseparate signals, on the assumption they are similar could also be usedto solve temporal problems. Finally, once temporal alignment isaccomplished, automatic amplitude adjustment could be performed on theoriginal sample to provide an optimal match between the compositesection containing the sample and the original sample.

It may be desirable to dynamically vary the encoding/decoding algorithmduring the course of encoding/decoding a signal stream with a givenwatermark. There are two reasons for dynamically varying theencoding/decoding algorithm.

The first reason for dynamically varying the encoding/decoding algorithmis that the characteristics of the signal stream may change between onelocality in the stream and another locality in the stream in a way thatsignificantly changes the effects that a given encoding algorithm mayhave on the perception of that section of the stream on playback. Inother words, one may want the encoding algorithm, and by implication,the decoding algorithm, to adapt to changes in the signal streamcharacteristics that cause relative changes in the effects of theencoding algorithm, so that the encoding process as a whole causes fewerartifacts, while maintaining a certain level of security or encoding agiven amount of information.

The second reason for dynamically varying the encoding/decodingalgorithm is simply to make more difficult attempts at decodingwatermarks without keys. It is obviously a more difficult job to attemptsuch attacks if the encoding algorithm has been varied. This wouldrequire the attacker to guess the correct order in which to use variousdecoding algorithms.

In addition, other reasons for varying the encoding/decoding algorithmsmay arise in the future.

Two methods for varying of the encoding/decoding algorithms according toembodiments of the present invention are described herein. The firstmethod corresponded to adaptation to changing signal characteristics.This method requires a continuous analysis of the sample windowscomprising the signal stream as passed to the framework. Based on thesecharacteristics, which are mathematically well-defined functions of thesample stream (such as RMS energy, RMS/peak ratio, RMS differencebetween samples—which could reflect a measure of distortion), a newCODEC module, from among a list of pre-defined CODECs, and thealgorithms implemented in them, can be applied to the window inquestion. For the purpose of this discussion, windows are assumed to beequivalent to frames. And, in a frame-based system, this is astraightforward application of the architecture to provide automatedvariance of algorithms to encode and decode a single watermark.

The second method for varying of the encoding/decoding algorithmscorresponds to increased security. This method is easier, since it doesnot require the relatively computationally-expensive process of furtheranalyzing the samples in a frame passed to the Framework. In thismethod, the Framework selects a new CODEC, from among a list ofpre-defined CODECs, to which to pass the sample frame as a function ofthe pseudo-random key employed to encode/decode the watermark. Again,this is a straightforward application of framework architecture whichprovides automated variance of algorithms to encode and decode a singlewatermark versus limitations evident in the analysis of a single randomnoise signal inserted over the entire content signal as proposed byDigimarc, NEC, Thorn EMI and IBM under the general guise of spreadspectrum, embedded signaling schemes.

It is important to note that the modular framework architecture, inwhich various modules including CODECs are linked to keys, provides abasic method by which the user can manually accomplish such algorithmicvariations for independent watermarks. The main difference detailedabove is that an automated method to accomplish this can be used withinsingle watermarks.

Automated analysis of composited copyrighted material offers obviousadvantages over subjective “human listening” and “human viewing” methodscurrently used in copyright infringement cases pursued in the courts.

What is claimed is:
 1. A method of encoding a digital watermark into asignal, comprising the steps of: selecting a sample window in thesignal; determining a quantization interval of the sample window,wherein the quantization interval can be used to quantize normalizedsample windows; normalizing the sample window to provide normalizedsamples, wherein the normalized samples conform to a limited range ofnormalized values which can be divided by the quantization interval intodistinct quantization levels; analyzing the normalized samples todetermine a quantization level; adjusting the quantization level of asample window based on an associated portion of the digital watermark;and de-normalizing the analyzed normalized samples.
 2. The method ofclaim 1, wherein the quantization interval and quantization levelscomprise nonlinear quantization information.
 3. The method of claim 2,wherein the nonlinear quantization information comprises one of blockfloating point system, floating point system, nonuniform compounding,adaptive delta modulation, adaptive differential pulse-code modulationand perceptual coding scheme information.
 4. The method of claim 2,further comprising the step of: pre-analyzing a digital sample toevaluate a substantially optimal location for the digital watermarkbased on a quantization envelope.
 5. The method of claim 2, furthercomprising the step of: randomly introducing noise significantly above aquantization amplitude on a window-by-window basis.
 6. The method ofclaim 5, wherein the noise is local dither.
 7. The method of claim 5,further comprising the step of: doping the signal to introduce a degreeof random errors that do not contain information associated with thedigital watermark.
 8. The method of claim 2, wherein the digitalwatermark is further encoded into the signal using a perceptual codingsystem.
 9. The method of claim 8, wherein the perceptual coding systemis associated with a compression scheme.
 10. The method of claim 9,wherein the compression scheme is one of MPEG-1 and AC-3.
 11. A methodof decoding a digital watermark from a signal, comprising the steps of:selecting a sample window in the signal; determining a quantizationinterval of the sample window, wherein the quantization interval can beused to quantize normalized sample windows; normalizing the samplewindow to provide normalized samples, wherein the normalized samplesconform to a limited range of normalized values which can be divided bythe quantization interval into distinct quantization levels; analyzingthe quantization level of the normalized samples to determine a portionof the digital watermark.
 12. The method of claim 11, wherein thequantization interval and quantization levels comprise nonlinearquantization information.
 13. The method of claim 12, wherein thenonlinear quantization information comprises one of block floating pointsystem, floating point system, nonuniform compounding, adaptive deltamodulation, adaptive differential pulse-code modulation and perceptualcoding scheme information.
 14. The method of claim 12, furthercomprising the step of: pre-analyzing a digital sample to evaluate asubstantially optimal location for the digital watermark based on aquantization envelope.
 15. The method of claim 12, further comprisingthe step of adding dither to the signal.
 16. The method of claim 15,wherein the dither is one of triangular probability density function(pdf), Gaussian pdf and rectangular pdf dither.
 17. The method of claim15, wherein the digital watermark is contained in a random-super-levelnon-subtractive dither.
 18. The method of claim 12, wherein the digitalwatermark is further encoded into the signal using a perceptual codingsystem.
 19. The method of claim 18, wherein the perceptual coding systemis associated with a compression scheme.
 20. The method of claim 19,wherein the compression scheme is one of MPEG-1 and AC-3.
 21. An articleof manufacture comprising a machine-readable medium having storedthereon instructions adapted to be executed by a processor, theinstructions which, when executed, result in the process: selecting asample window in the signal; determining a quantization interval of thesample window, wherein the quantization interval can be used to quantizenormalized sample windows; normalizing the sample window to providenormalized samples, wherein the normalized samples conform to a limitedrange of normalized values which can be divided by the quantizationinterval into distinct quantization levels; analyzing the normalizedsamples to determine a quantization level; adjusting the quantizationlevel of a sample window based on an associated portion of the digitalwatermark; and de-normalizing the analyzed normalized samples.
 22. Anarticle of manufacture comprising a machine-readable medium havingstored thereon instructions adapted to be executed by a processor, theinstructions which, when executed, result in the process: selecting asample window in the signal; determining a quantization interval of thesample window, wherein the quantization interval can be used to quantizenormalized sample windows; normalizing the sample window to providenormalized samples, wherein the normalized samples conform to a limitedrange of normalized values which can be divided by the quantizationinterval into distinct quantization levels; analyzing the quantizationlevel of the normalized samples to determine portion of the digitalwatermark.